1. Field of the Invention
The present invention relates generally to image/video display and processing systems, and in particular, to a method, apparatus, and article of manufacture for viewing and tracking the progress and state of elements in a clip.
2. Description of the Related Art
A clip is a sequence of images or frames loaded from an external device such as a VTR (video tape recorder), stored in digital format, and displayed in a clip library and on a desktop. Clips may be edited, processed, and recorded to/on external devices. However, the prior art fails to provide an efficient and effective mechanism for tracking the progress and state of individual clip elements/operations in a collaborative environment. Instead, such information must be tracked manually. Thus, the prior art fails to provide the capability to determine whether an important edit (e.g., one that must be completed before others can take place) is going to be completed when needed. Such prior art problems may be better understood with a description of prior art clips and clip processing.
Post production of sequences for film and video is an extremely complex undertaking. Each clip may have dozens of separate elements/operations that need to be combined/composited, each individual element requiring significant effort on the part of a digital artist. For example, to composite a moving car onto a new background may require a mask for the car be cut and perfected for every frame of the sequence.
Professional compositing software helps digital artists and managers visualize the process of clip development by building a “dependency graph” of elements and effects for each sequence. In such software, operators, tasks, or modules/functions perform a variety of functions with respect to a clip or source. For example, a module may perform compositing, painting, color correction, degraining, blurring, etc. A dependency graph of such software may also be referred to as a process tree or a schematic view of a process tree. Such a tree illustrates the elements and effects needed to complete a sequence of functions and identifies the hierarchy that shows the relationship and dependencies between elements. Accordingly, as the end-user interacts with media and tools in a schematic view, the system constructs a graph representing the activity and the relationships between the media and tools invoked upon them. The purpose of the dependency graph is to manage and maintain the relationships between the nodes representing the media and the tools.
FIG. 1 illustrates a process tree in a schematic view. A schematic view 100 uses node(s) 102-104 to represent all of the operators in a workspace performed on one or more clips 106-110, and line(s) (edges) 112-118 between nodes clips 106-110 and nodes 102-104 to indicate the flow of image data between the operators. The output of one node 102 is the source for the next node 104. As a result clip 104 or other branch (e.g., paint or color correction projects 102) is created, the process tree indicating the flow of data through the nodes 102-104 is built automatically. As illustrated in FIG. 1, various source nodes in the form of clips 106-110 are utilized. As illustrated, front clip 106, back clip 108, and matte clip 110 may be utilized by color correction node 102. The color correction node 102 performs an operation (i.e., a color correction to the clips 106-110) on the clips 106-110 to produce a result/output 104.
As used herein, an operator (e.g., color correction operator 102) is the most common node. Alternative nodes may be used to connect two nodes. The process tree (such as the process tree illustrated in FIG. 1) can be used to manage operators and nodes. For example, users may add operators (e.g., a paint operation) to a branch, change the process order of operators 102-104, and navigate easily between operator controls. Further, if an operator is added to a layer in a schematic view 100, the operator will appear as a node (e.g., the color correct node 102). The same operator node may also appear in other workspace environments (e.g., a workspace panel). Further, in the schematic view, particular operators may be turned on/off as desired simply by clicking on the icon for that operator.
Thus, as described above, the dependency graph is the record of end-user activity in the creative application. The dependency graph is constructed by the system dynamically, consists of various nodes and connections, and is represented in the user interface as the schematic view 100. As the end-user manipulates media by editing, applying tools, etc., the creative application constructs a dependency graph reflecting the structure of the composition. For example, when the end-user drops a keyer into the schematic 100, the system creates an instance of a tool node for the keyer in the dependency graph 100.
Thus, the dependency graph consists of tool nodes, link nodes, and occasionally, group nodes. Tool nodes represent the invocation of a tool and its parameters. Link nodes connect the graph 100 to the published result of another composition and group nodes can be created by an end-user to arrange nodes together in the schematic 100 for visual clarity.
While the above prior art schematic view 100 and operations maybe used to modify and edit an image/clip on a single workstation, the prior art has many limitations. In particular, prior art software fails to provide any mechanism that helps manage the process of tracking the progress and state of each clip element across multiple users, allowing a lead artist to easily understand the progress and state-of-completeness of a given sequence. Instead, artists/designers must track such information manually while in a large collaborative environment. Accordingly, there is no efficient and useful means (other than visual inspection) for an art director to know whether an important edit (one that must be completed before others can take place) is going to be completed when needed.
In addition to the above, it is noted that many digital artists create multiple variations of an individual clip element that are combined into a single collaborative project. Such individual variations must be managed. It is desirable to manage the variations in a manner that allows the changes to be easily “tried out” in the compositing software. The prior art fails to provide such capabilities.
Prior art techniques attempt to address the deficiencies described above through various work-arounds. For example, prior art software may allow a user to re-create the dependency tree/graph of a render pipeline, and to analyze it for critical-path items. However, such an approach fails to provide a means to manage multiple clip variations, to generate a variety of usage reports, or to perform any asset management.
In addition to the above, the use of a dependency graph in a collaborative environment has many problems. For instance, the prior art fails to provide a methodology or system to perform collaboration through the use of a dependency graph. For example, the prior art primarily relies on two different models: (1) file-based model; or (2) hidden file/application based model.
In a file based model, a user opens a file (e.g., a project, scene, word processing document, drawing, etc.) and while the file is open, the user has read/write access to the file. Most prior art applications do not prevent other applications from opening the same file. In this regard, to perform versioning, the user is required to perform a “save as” or similar type of operation. Thus, the file-based model is similar to a check-in/check-out system where the application checks in or out a given application and no other users can access the file while checked-out. Nonetheless, while some applications may have a built-in file sharing capability, the capabilities are very limited and most prior art applications have no mechanism for preventing multiple users from accessing the same file. Further, the information within the single file is difficult if not impossible to leverage. For example, detailed information such as the effect, degraining, color correction, etc. being performed are not available or accessible outside of the application itself.
In a hidden file/application based model, when an application is executed, a hidden file is accessed such that the data viewed in the application is carried over from the prior execution of the application. For example, a data structure that is associated with a particular application may be opened at run time and load the application with the data viewed during the prior execution of the application. The hidden file is accessible (e.g., via read/write access) to the single instance of the application during run time. Further, when the application is closed, the hidden file is updated and saved. Thus, the data stored by the application appears to the user as if it were a constant save. However, similar to the file-based model, the application-based model is not intended for use in a collaborative environment. In this regard, only one instance of the application can open or use the data structure/hidden file at any given time. Thus, there is no mechanism for many people or instances of the application to access or write to a single hidden file/data structure.
In view of the above, under the hidden file/application based model, a separate file could be used for every computer/instance of the application for version control. Further, the sequence or set of files could be stored on a central server. However, the prior art fails to provide any ability to utilize/access the information contained within multiple instances of the hidden files. For example, in a spreadsheet or accounting system, the prior art would allow multiple different instances of the accounting system to open up multiple different hidden files stored on a server. However, the prior art would not permit the user to see the end balance for every user of the system (or for the multiple applications). Thus, the prior art fails to provide any application specific data knowledge outside of the application itself. Similarly, the specific effects created by artists in a collaborative scene are not available or accessible in a single application.
In view of the above, prior art applications fail to provide detailed creative/non-creative based information generated by/during use of the application once an application has completed execution. For example, prior art applications may provide naming convention based information (wherein the name of the file is used to derive information) such as the name of a file, the last modified date of a file, the owner of the file, or other information that is user dependent. However, if the naming convention or the actual name of the file is changed by a user, the information is lost or not easily determinable.
An example of the failure to provide application specific knowledge exists with compositing/editing software. Suppose a city is being created with 200 buildings and a superhero flying through the city. An application would create the city with references to all of the buildings. The details and artistic attributes of each building could be farmed out to separate artists. As each artist adds definition, multiple versions of that single building are created/generated. In the prior art, the city is manually managed by people. In this regard, the person has the responsibility to manually track the different versions of each building across all artists. If the wrong version/building is loaded, the result could be costly (e.g., based on the re-rendering of the multiple frames that include the building). The prior art may elect to manage such a city using a particular naming convention. However, as described above, such a naming convention would not provide sufficient information and individual artists could change the name and the manager may not be aware of the change. Thus, prior art facilities have no mechanism for determining/maintaining organization/knowledge of the desired information for a particular shot, sequence, project, or movie.
The city/building example above also illustrates the failure of the prior art to consider that dependencies may frequently change (especially in a collaborative environment). In this regard, references to the buildings may change frequently as people create new versions. In the prior art, a snapshot at a given time is taken and all dependencies (e.g., in a dependency graph as described above) are stored for that snapshot. However, as the information and data changes, the prior art fails to update the references. Thus, while data is maintained for a particular snapshot in time, such a snapshot may not be updated nor accurately reflect information. Accordingly, there is no mechanism for maintaining knowledge of the state of every single element or work in a collaborative environment.
In addition to the snapshot information, non-creative information used in the application is not stored or viewable by the application or outside of the application. For example, the time spent by particular artists, the time spent rendering, the amount of storage space consumed, etc. are not collectable or viewable in the prior art. Instead, only peripheral user-based information such as the date changed is available/accessible. Further, color correction, setup information, information relevant to the creative aspects of the application, cross-dependencies (e.g., on a project basis), and other information are not available nor exposed in the prior art.
The failure to maintain accurate information and the inability to expand in a collaborative environment also present multiple additional issues. For example, scripting applications in the prior art may be utilized to perform various effects on data. However, without error checking and the ability to exclude access to certain users (or to maintain a system for reading/writing files), such prior art scripting applications often override work that an artist has performed. In this regard, the prior art fails to maintain the knowledge of whether a particular artist has a file open or if a certain version is required.
In view of the above, what is needed is a mechanism to effectively and efficiently process, track, maintain, and manage clips and/or elements within clips.